Skip to content

test: Add E2E unit tests for CLP connector. - #54

Merged
anlowee merged 19 commits into
y-scope:release-0.293-clp-connectorfrom
anlowee:xwei/presto-e2e-unit-test
Sep 3, 2025
Merged

test: Add E2E unit tests for CLP connector.#54
anlowee merged 19 commits into
y-scope:release-0.293-clp-connectorfrom
anlowee:xwei/presto-e2e-unit-test

Conversation

@anlowee

@anlowee anlowee commented Aug 15, 2025

Copy link
Copy Markdown

Description

This PR updates enables the end-to-end testing functionality to CLP connector. The end-to-end testing feature is bundled for all native connectors such as Hive, Iceberge, and TPCH, so it is good for CLP to support it as well. Another important benefit is that it can save a lot of time setup the end-to-end testing environment and also ensure the we don't break functionality for each PR.

The first change is instead of ClpMetadataDbSetUp we implemented another ClpMockMetadataDatabase. This class provides a builder style way to setup the metadata database. For consistency, besides ArchivesTableRows we also defined ColumnMetadataTableRows and DatasetsTableRows to represent the three tables that will be used by CLP connector, and ClpMockMetadataDatabase has some methods to register the necessary information by the data whose types are these three classes.

The second change is in the presto-native-execution module. Similar to TestPrestoNativeHiveGeneralQueries and TestPrestoNativeIcebergGeneralQueries, we implemented TestPrestoNativeClpGeneralQueries. The major difference between the last one and the former two is that the entire framework is designed for those have both Java worker implementation and Pretissimo implementation, so they can use a Java coordinator and a Java worker (expected query runner) to compare with the Java coordinator and a Pretissimo worker (actual query runner) to do the unit testing. Also the native worker creation is hardcoded to create hive connector, so we also create a function getExternalClpWorkerLauncher to create our own worker.

This PR only adds a select * query for sanity check. Later, we should integrate the TestClpMetadata, TestClpSplit and TestClpQueryBase. Eventually, we should get rid of ClpMetadataDbSetUp. The tests related to the metadata database should be all integrated into the end-to-end testing framework. The end-to-end testing framework should also cover some basic queries to validate the functionality. For presto-clp tests, we should only keep those testing without the need of metadata database, for example the pushdown generation etc.

Checklist

  • The PR satisfies the contribution guidelines.
  • This is a breaking change and that has been indicated in the PR title, OR this isn't a
    breaking change.
  • Necessary docs have been updated, OR no docs need to be updated.

Validation performed

Passed the CI (the unit test class will be automatically added into CI):
image

Summary by CodeRabbit

  • Tests

    • Added end-to-end CLP integration tests and test utilities for native execution, including a mock metadata DB, table-population helpers, CLP query-runner harness, and an external CLP worker launcher; validates query execution and exact result values.
  • Chores

    • Centralized CLP dependency management and added runtime/test-jar/test-scoped CLP dependencies across modules to support testing.
  • Notes

    • No user-visible API or runtime behavior changes.

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants